Device for enhancement of language processing in autism spectrum disorders through modifying the auditory stream including an acoustic stimulus to reduce an acoustic detail characteristic while preserving a lexicality of the acoustics stimulus

ABSTRACT

Methods and devices can enhance language processing in an autism spectrum disorder (ASD) individual through auditory manipulation of an auditory stream. The auditory stream is received and includes an acoustic stimulus perceptually representing an object. An acoustic manipulation parameter for a predetermined acoustic detail characteristic is selected. The predetermined acoustic detail characteristic is associated with the ASD individual and is based on a measured language processing capability of the ASD individual. The auditory stream is modified based on the selected parameter, to reduce the predetermined acoustic detail characteristic while preserving a lexicality of the stimulus, such that the reduced acoustic detail characteristic enhances perception of the object by the ASD individual even when the stimulus includes two or more acoustically distinct stimuli each perceptually representing the object. The modified auditory stream is output to the ASD individual via at least one loudspeaker.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Grant Nos.R01DC008871 and R01HD073258 awarded by the National Institutes ofHealth. The Government has certain rights in the invention.

CROSS REFERENCE TO RELATED APPLICATION

This application is the U.S. national phase entry of InternationalApplication No. PCT/US2014/050893, filed Aug. 13, 2014, which claimspriority to U.S. Provisional Application Ser. No. 61/865,293 entitledDEVICE FOR ENHANCEMENT OF LANGUAGE PROCESSING IN AUTISM SPECTRUMDISORDERS AND RELATED LANGUAGE IMPAIRMENTS, filed Aug. 13, 2013. Thecontents of International Application No. PCT/US2014/050893 and U.S.Provisional Application Ser. No. 61/865,293 are incorporated fullyherein by reference.

FIELD OF THE INVENTION

The present invention relates to language processing. More particularly,the present invention relates to devices and methods for languageprocessing enhancement for individuals with language impairmentdisorders based on reduction of acoustic detail in the auditory stream.

BACKGROUND OF THE INVENTION

Autism spectrum disorder (ASD) refers to a set of developmentaldisorders which are identified in children and continue into adulthood;characteristic symptoms of ASD involve deficits in language andcommunication, in addition to difficulties with socialintegration/interaction, and repetitive movements. The prevalence of ASD(e.g., about 1 out of 110 individuals have at least one spectrumdisorder), with a large number of ASD children being profoundly impairedin language (e.g., with about 40% without any linguistic behavior).

Some children on the autism spectrum (and possibly relatedneuropsychiatric disorders such as central auditory processing delay(CAPD)) have a difficulty in “abstracting” or forming perceptual objectsfrom acoustically distinct, but conceptually identical stimuli. Anexample would include difficulty in the encoding of the same word spokenby different speakers. This difficulty in abstracting has implicationsfor ASD individuals with respect to their systems of word (lexical)processing and representation, including the speed and success ofactivation of abstract lexical representations.

SUMMARY OF THE INVENTION

The present invention relates to a method of auditory manipulation of anauditory stream for enhancement of language processing in an autismspectrum disorder (ASD) individual. A processor receives the auditorystream, where the auditory stream includes an acoustic stimulusperceptually representing an object. An acoustic manipulation parameteris selected for a predetermined acoustic detail characteristic. Thepredetermined acoustic detail characteristic is associated with the ASDindividual and based on a measured language processing capability of theASD individual. The processor modifies the auditory stream based on theselected acoustic manipulation parameter. The modification to theauditory stream reduces the predetermined acoustic detail characteristicwhile preserving a lexicality of the stimulus, such that the reducedacoustic detail characteristic enhances perception of the object by theASD individual even when the stimulus includes two or more acousticallydistinct stimuli each perceptually representing the object. The modifiedauditory stream is output to the ASD individual via at least oneloudspeaker.

The present invention also relates to a device for auditory manipulationof an auditory stream for enhancement of language processing in anautism spectrum disorder (ASD) individual. The device includes an audioinput interface, a non-transitory tangible storage device, an acousticdetail manipulation unit and at least one loudspeaker. The audio inputinterface is configured to receive the auditory stream, where theauditory stream includes an acoustic stimulus perceptually representingan object. The storage device is configured to store acousticmanipulation parameters for a predetermined acoustic detailcharacteristic. The predetermined acoustic detail characteristic isassociated with the ASD individual and based on a measured languageprocessing capability of the ASD individual. The acoustic detailmanipulation unit is configured to: select an acoustic manipulationparameter among the stored acoustic manipulation parameters for thepredetermined acoustic detail characteristic and modify the auditorystream based on the selected acoustic manipulation parameter. Themodification to the auditory stream reduces the predetermined acousticdetail characteristic while preserving a lexicality of the stimulus,such that the reduced acoustic detail characteristic enhances perceptionof the object by the ASD individual even when the stimulus includes twoor more acoustically distinct stimuli each perceptually representing theobject. The at least one loudspeaker is configured to output themodified auditory stream to the ASD individual.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may be understood from the following detailed descriptionwhen read in connection with the accompanying drawings. It is emphasizedthat, according to common practice, various features/elements of thedrawings may not be drawn to scale. On the contrary, the dimensions ofthe various features/elements may be arbitrarily expanded or reduced forclarity. Moreover, in the drawings, common numerical references are usedto represent like features/elements. Included in the drawings are thefollowing figures:

FIG. 1 is a functional block diagram of an example auditory manipulationdevice for language processing enhancement, according to an embodimentof the present invention;

FIG. 2 is a flowchart of an example method for calibration of anauditory manipulation device, according to an embodiment of the presentinvention;

FIG. 3A is a flowchart diagram of an example method of auditorymanipulation for enhancement of language processing, according to anembodiment of the present invention;

FIG. 3B is a flowchart diagram of an example method of auditorymanipulation for enhancement of language processing, according toanother embodiment of the present invention;

FIG. 4A is a spectrogram illustrating an example uniqueness point of theword “cat,” according to an embodiment of the present invention; and

FIG. 4B is a graph of an example activation of the word “cat” as afunction of time, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

As discussed above, ASD individuals have difficulty abstractingperceptual objects from acoustically distinct but conceptually identicalstimuli. This difficulty may be caused by impairment in the basicbuilding blocks of language function, by the creation of redundantrepresentations of the same “object”, colored by too much attention toirrelevant acoustic detail.

Evidence for such failure may be seen in electrophysiologic responses(e.g., recorded by magnetoencephalography (MEG), a type of functionalbrainwave mapping which allows depiction of brain function in space andtime) indicating lack of “clustering” of responses around perceptualobjects, a heightened sensitivity to acoustic (but not linguistic)differences and a general delay processing auditory input (secondary toexerting too much effort processing minor acoustic differences). Primingeffects (both electrophysiologically and behaviorally) would alsoindicate heightened attention to “irrelevant” details, precludingperceptual object formation.

In general, spoken words may contain too much sound information; thatis, more information than is needed to “decode” the intended “object” ormeaning. Sometimes, that extra detail helps in higher order distinctions(e.g., was it a male or a female speaker . . . ? were they angry . . .?) and sometimes it just fluctuates (speaker to speaker, or even day today). Typical development allows a tolerance to such “minor” acoustic orsound differences and permits clustering of different “sound” eventsinto the same “word”. By over-reliance on sound details, or by inabilityto cluster, or both, the ASD brain may fail to tolerate sounddifferences that are essentially irrelevant to meaning extraction.

It may be appreciated that “relevance” may be in the eye (ear) of thebeholder—generally concepts of “irrelevance” are referred to herein asfactors not necessary to relay or impart object recognition,representation and ultimately meaning.

Every sound, even a speech sound like a word, contains acoustic featuresor details. These include pitch (frequency content), harmonics,richness, etc. These are physical and can be measured directly (orsynthesized by a computer, for example). In speech they might reflectspeaker (male: low pitch, female: high pitch), intonation (angry/sad) orurgency (speed of transients etc.); or they might just be the result offluctuations in the speech apparatus. What matters is that the same“word” such as “cat” could be uttered by different speakers, withdifferent intonations or different urgency (or simply on differentoccasions) and each utterance would have slight-moderate “acousticdifferences”. However, to a “normal ear/brain”, all would be perceivedas a single “object”, namely a cat. A linguistic difference might beembodied in a special case of an acoustic difference, where thedifference is sufficient to transform the perceived object from e.g.“cat” to “cap”, with associated different meaning. The ASD brain mayfail to appropriately “cluster” across acoustic differences and thushears different versions of “cat” as multiple different objects (andthus may not arrive at the intended meaning either as quickly oraccurately, in essence because of giving too much weight or significanceto “acoustic differences” that are irrelevant to arriving at anappropriate representation or meaning).

Typical electrophysiologic responses to words (as opposed to non-wordslike “blik”, for example) are associated with spectrotemporal features(specifically decreased oscillatory power in the 5-15 Hz band, afraction of a second after the word is recognized). Responses in ASD maybe delayed or attenuated. This may be scanned and detected using MEG.

Lexical processing in typically developing (TD) individuals involvesboth abstract and episodic components. Every single time a word (e.g.,cat) is heard, it has physically distinct properties: individual speakervoice, speaker gender, differences in prosody and volume, etc., makingevery spoken word a unique event. The TD brain is able to abstract fromall of these distinct acoustic objects (CAT₁ . . . CAT_(n)) a singlelexical representation of “cat.” The TD brain does this by factoring outirrelevant phonetic detail, and working with a phonologicalrepresentation /kaet/ that serves as the basis for lexicalrepresentation. At the same time, the TD brain might also be sensitiveto some episodic properties of words, so that some aspects of individualinstances of heard words (e.g., information about speaker, or speechrate, or phonetics) are represented in memory as well, where they havesome effect in linguistic processing. The abstract component tends to bemore important for linguistic representation and processing of words,because, when cat is heard, the lexical-semantic (and other, e.g.,syntactic) properties of this lexical item are activated. For ASD, apopulation with a general problem with abstraction, and over-attentionto low-level detail, the episodic aspect of lexical processingpredominates over or compensates for the abstract system, withconsequences for the linguistic system.

Recognizing an auditorily presented word involves a number of distinctcomputations in the brain. These computations include: converting theacoustic signal into phonological representations (linguisticrepresentations of sound); matching the incrementally derived soundrepresentations with words in the hearer's brain (e.g., after hearingthe first two segments of cat, [kae], the brain has partially activatedcat, cap, cab, etc.); and selecting one of the activated candidates asthe winner, when enough of the word's sound form is known so that thiscan be done. For example, after the final segment [t] of cat is heard,the word cat is fully activated in the hearer's brain. This view oflexical access involves three key notions that play a role in ourinvestigation of the ASD brain. Lexical access involves the onset oflexical search/activation (lex-search), a recognition point at which thebrain identifies the target word (recog-point) and a uniqueness point inthe acoustic signal of the word at which that word becomes uniquelyidentifiable, such as the final [t] of cat).

FIGS. 4A and 4B illustrate the lexical access described above for theword “cat.” FIG. 4A is an example spectrogram of a speech signalincluding the word “cat,” and FIG. 4B is a graph of an example lexicalaccess of the word “cat” as a function of time. FIG. 4A is time-lockedwith the lexical brain activity shown in FIG. 4B. As shown in FIGS. 4Aand 4B, the uniqueness point of cat is determined by the final consonantsound [t]. The lexical system (in FIG. 4B) is able to raise theprocessed word above the threshold only after the uniqueness point hasbeen processed.

FIG. 4B shows increasing activation (schematized on the y-axis) oflexical access mechanisms in the brain over time. The incoming speechsignal triggers increased auditory processing. As the acoustic signal isconverted into phonological representations, activation devoted to thelex-search increases, with onset immediately after the acoustic signalis recognized as speech (region 1). The lexical search produces a numberof candidate lexical items compatible with incrementally-processed input(region 2). When the uniqueness point of the word arrives, the lexicalsystem raises the activation of the winning candidate cat abovethreshold (region 3), the recognition point. The word cat is then fullyactivated.

The ASD brain is delayed with respect to early auditory responses, whichresults in a slower onset of auditory processing. Thus, the lex-searchcomponent is delayed and/or less robust. The ASD brain's inability toabstract/concentrate on relevant phonetic detail may also make it unableto recognize different phonetic tokens of the same phonological sequenceto be recognized as the same sequence. An immediate consequence is thatit is not possible to produce candidates for the lex-search, and toreach a threshold for a single word (recognition-point) with the speedand accuracy of TD individuals.

A priming effect (both behaviorally and in terms of quantitativefeatures of the MEG recoding, described above) can be observed forexample by repetition. For example, “cat” after “cat” is processedfaster and has muted electrophysiologic responses compared to “cat”after “truck”. Of note, “cat” after “dog” is a bit of a middleground—because of their relatedness—called “semantic priming”. However,if tokens of “cat” are perceived as distinct objects because of too muchemphasis on “irrelevant” “acoustic details”, then priming effects couldbe lost in ASD. In fact, the degree of priming effect attenuation mightindeed index the degree of perceptual clustering deficit, and ultimatelylanguage impairment.

Irrelevant acoustic details may be generally defined as “acousticdifferences” that should typically detract from the appreciation of theword (and its representation, or meaning) but that might cause separateclassification in the ASD brain which “cannot see (hear) the forest forthe trees”. A simple example would be pitch (reflecting speaker, but notchanging meaning).

Aspects of the present invention relate to acoustic manipulation of theauditory stream (e.g., such as via a hearing aid-type device), toimprove perceptual object formation from acoustically distinct, butconceptually identical stimuli. Example methods and devices relate toauditory manipulation of an auditory stream for enhancement of languageprocessing in an autism spectrum disorder (ASD) individual. The auditorystream includes an acoustic stimulus perceptually representing anobject. An acoustic manipulation parameter may be selected for apredetermined acoustic detail characteristic. The predetermined acousticdetail characteristic is associated with the ASD individual and is basedon a measured language processing capability of the ASD individual. Anacoustic detail manipulation unit may modify the auditory stream basedon the selected acoustic manipulation parameter. The modification to theauditory stream reduces the predetermined acoustic detail characteristicwhile preserving a lexicality of the stimulus, such that the reducedacoustic detail characteristic enhances perception of the object by theASD individual even when the stimulus includes two or more acousticallydistinct stimuli each perceptually representing the object. The modifiedauditory stream may be output to the ASD individual via at least oneloudspeaker.

Referring to FIG. 1, a functional block diagram of an example auditorymanipulation device, designated generally as device 100, is shown.Device 100 may manipulate auditory stream 132, by reducing predeterminedacoustic detail characteristics of auditory stream 132. The modifiedaudio signal 134 (with reduced acoustic detail) may enhance languageprocessing for ASD individuals, by improving perceptual object formationfrom acoustically distinct but conceptually identical stimuli). Thepredetermined acoustic detail characteristics (also referred to hereinas predetermined characteristics) to be reduced may be associated with alanguage processing capability of a user of device 100. Thepredetermined characteristics (and device 100 in general) may becalibrated based on the user's language processing capability (describedfurther below with respect to FIG. 2).

Device 100 may include microphone 102, audio input interface 104,acoustic detail manipulation unit 106, amplifier(s) 108, loudspeaker(s)110, controller 112, storage 114, user interface 116 and power supply118. Microphone 102, audio input interface 104, acoustic detailmanipulation unit 106, amplifier(s) 108, loudspeaker(s) 110, controller112, storage 114, user interface 116 and power supply 118 may be coupledtogether via a data and control bus (not shown). Power supply 118 mayinclude any suitable power supply (such as a battery) capable ofpowering components of device 100. Although not shown, device 100 mayinclude a data communication interface, for wired or wirelesscommunication with a remote device. Although not shown, device 100 maybe coupled to a remote location, for example via a global network (i.e.,the Internet).

In some examples, device 100 may be configured as a hearing device,including, without being limited to, a behind-the-ear (BTE) type device,an in-the ear (ITE) type device, an in-the-canal (ITC) device or as awearable hearing device. Loudspeaker(s) 110 may be configured to bepositioned in the user's ear(s) (such as in an earpiece). In someexamples, device 100 is a monaural device, and includes one loudspeaker110 (and one corresponding amplifier 108). In other examples, device 100is a binaural device, and includes two loudspeakers 110 (and twocorresponding amplifiers 108). In some examples, device 100 may includecomponents (such as microphone 102, audio input interface 104, acousticdetail manipulation unit 106, amplifier 108, controller 112, storage114, user interface 116 and power supply 118) that may be configured inone housing unit. The housing unit may be worn on any part of the user'sbody (for example, on the user's belt, in a pocket, as a pendent aroundthe user's neck, on a wristband on the user's wrist, etc.).Loudspeaker(s) 110 may be formed in an earpiece(s) positioned in theuser's ear(s) (such as at the entrance to the ear canal or in the earcanal) and coupled to the housing unit.

Microphone 102 is configured to receive an input acoustic signal 128from the ambient acoustic environment, such as from an individualproximate to microphone 102. Microphone 102 may include any suitabletransducer capable of converting the input acoustic signal 128 to an(electronic) audio signal 129.

In some examples, audio input interface 104 is configured to receiveremote audio signal 130 (an electronic signal) from a remote device suchas a phone (including a mobile phone), a computer, a television, anaudio playback device, a video playback device (having audio capability)or any device capable of providing remote audio signal 130. The remotedevice (not shown) may be wired or wirelessly coupled to audio inputinterface 104. Audio input interface 104 may also include ananalog-to-digital converter (ADC) for converting audio signal 129 (frommicrophone 102) to a digital signal. In general, audio input interfacereceives audio signal 129 and/or remote audio signal 130 and convertsthese signals to an auditory stream 132. Audio input interface 104 mayinclude any hardware and/or software components to receive and/or modifyaudio signal 129 (and/or audio signal 130) to form auditory stream 132in a format suitable for processing by acoustic detail manipulation unit106.

Acoustic detail manipulation unit 106 is configured to receive auditorystream 132 and to provide a modified audio signal 134. Modified audiosignal 134 may be acoustically manipulated by one or more components ofacoustic detail manipulation unit 106, to reduce one or morepredetermined acoustic detail characteristic(s) of auditory stream 132,based on the language processing capability of the user. Acoustic detailmanipulation unit 106 may include one or more of: a detail removalfilter 120, a noise generator 122, a speech recognizer 124 and a speechsynthesizer 126. In some examples, acoustic detail manipulation unit 106may include one component (such as detail removal filter 120 or noisegenerator 122). In other examples, acoustic detail manipulation unit 106may include only speech recognizer 124 and speech synthesizer 126. Thecomponents to be included in acoustic detail manipulation unit 106 maybe selected upon calibration of device 100 for the user (describedfurther below with respect to FIG. 2). Acoustic detail manipulation unit106 may include any suitable software and/or hardware components tomanipulate auditory stream 132 to reduce the predetermined acousticdetail characteristic(s). The predetermined acoustic detailcharacteristic(s) may be associated with parameters selected (and storedin storage 114) for detail removal filter 120, noise generator 122,speech recognizer 124 and/or speech synthesizer 126. The modification ofauditory stream 132, in general, reduces the predetermined acousticdetail characteristic(s) while preserving a lexicality of the degradedstimulus (i.e., the modified stimulus still sounds like a word).

Detail removal filter 120 may be configured to receive auditory stream132, and to filter or smooth auditory stream 132 such that all (or asuitable portion of) “extraneous” detail is removed. The filtered signalfrom detail removal filter 120 may form modified audio signal 134.Detail removal filter 120 may include a filter having a predeterminedfilter characteristic (such as a low-pass filter, a band-pass filter, ahigh-pass filter, etc.). For example, a majority of spectral energy inhuman speech is in the range of 100 Hz to 1 kHz. Accordingly, in anexample embodiment, a band-pass filter which passes frequencies in the20 Hz to 20 kHz range may be used.

The word “extraneous” relates to details that are not relevant (and mayindeed be distracting to ASD) for the purposes of identifying the“object” which the uttered sound (word) represents (e.g., intonation,frequency dynamics, etc.). An example filtering approach may remove highfrequency sibilants, etc., that might convey acoustic “attack” (onsetabruptness) or intonation, but that should prevent definition of aunique representation. Such an approach is similar to a low-pass filterwhich filters speech while maintaining the essence of the conversation.

Filter parameters (such as the cutoff frequency, the center frequency,the filter type) of device 100 may be tuned to the individual user. Assuch, a calibration may be performed on device 100 to adjust the filterparameters to the user, prior to its use. In addition, the calibrationmay also provide a quantitative index of therapeutic intervention.Filter parameters and the quantitative index may be further adjustedover time (e.g., the index may be reduced indicating a successfulprogress).

As another example, detail removal filter 120 may be configured toreceive auditory stream 132 having a first sampling rate, and tosub-sample auditory stream 132 with a predetermined second samplingrate, such that the second sampling rate (of filter 120) is less thanthe first sampling rate (of auditory stream 132). The predeterminedsecond sampling rate may be selected to reduce the predeterminedacoustic detail characteristics, which may be calibrated to the user.The sub-sampled signal from detail removal filter 120 may form modifiedaudio signal 134.

Noise generator 122 may be configured to receive auditory stream 132 andapply (i.e., add) noise having a predetermined noise characteristic toauditory stream 132. The predetermined noise characteristic may beselected to saturate acoustic details of auditory stream 132, such thatthe listener focuses only on the core auditory input. The noise-addedsignal from noise generator 122 may form modified audio signal 134.

For example, white noise (i.e., having a uniform broad band powerspectral density) or colored noise (e.g., pink noise having a lowfrequency emphasis, with a power spectral density inversely proportionalto frequency) may be added to auditory stream 132. White noise may beapproximated, for example, by a pseudorandom noise generator. While thenoise generally reduces sensitivity (commonly thought of as“intelligibility”), the noise may mask the extraneous features ofauditory stream 132, thus allowing the essence of the conversation to beappreciated (to force a desired clustering).

The predetermined noise characteristic may include a spectraldistribution (such as white noise or pink noise) and/or an amplitude ofeach component of the spectral distribution. The predetermined noisecharacteristic may be tailored to the individual according to theirhearing (audiology/audiometric testing), brain responses or throughtheoretical personalization. Similar to the predetermined filterparameters of detail removal filter 120, the predetermined noisecharacteristic may also be monitored and/or adjusted over time.

Speech recognizer 124 may be configured to receive and apply speechrecognition processing to auditory stream 132, to form a textrepresentation of auditory stream 132. Speech synthesizer 126 may beconfigured to receive the recognized speech (from speech recognizer 124)and convert the recognized speech to a single speech production voicevia speech synthesis processing. The speech recognition and speechsynthesis processing by respective speech recognizer 124 and speechsynthesizer 126 may be performed in real time. The speech productionvoice produced by speech synthesizer 126 may have predetermined speechcharacteristics that are tuned (i.e., calibrated) to the individual.Thus, auditory input from several different speakers may be recognizedas a same perceptual object (by speech recognition) and converted to asingle speech production voice (having the same predetermined speechcharacteristics regardless of the acoustic characteristics of thespeakers). The speech production voice from speech synthesizer 126 mayform modified audio signal 134.

Speech recognizer 124 may use any suitable speech recognition technique(such as used by automated dictation systems) to recognize multiplevoices (i.e., voices from different individuals) and distil the essenceof each voice into text. Example speech recognition techniques include,without being limited to, acoustic modeling and/or language modelingtechniques, including Hidden Markov Models (HMMs), and neural networks.Speech synthesizer 126 may interpret text (from speech recognizer 124)and reproduce human speech (but always with the same pitch, intonationetc., according to the predetermined speech characteristics). Thus,speech synthesizer 126 does not introduce variability in acousticdetails associated with a particular word. Accordingly, reproducibleword sounds may be produced by loudspeaker 110 in response to variableword sounds captured by microphone 102 or received from a remote device.

Similar to the filter parameters of detail removal filter 120, thepredetermined speech characteristics of speech synthesizer 126 may becalibrated to the individual user. A quantitative index of therapeuticintervention may be determined during calibration. Similar to the filterparameters, the predetermined speech characteristics and quantitativeindex may also be monitored/adjusted over time (e.g., the index may bereduced indicating a successful progress).

In some examples, both detail removal filtering and noise generation viarespective detail removal filter 120 and noise generator 122 may beapplied to auditory stream 132. In some examples, the speech synthesizedsignal (from speech synthesizer 126) may be applied to detail removalfilter 120 and/or noise generator 122, for further acoustic detailremoval. The selection of components of acoustic detail manipulationunit 106 may be tailored to the user's language processing capability.

Amplifier(s) 108 may be configured to receive and amplify modified audiosignal 134 from acoustic detail manipulation unit 106, to form amplifiedsignal 136. Amplifier(s) 108 may include any hardware and/or softwarecomponents to amplify modified audio signal 134, based on apredetermined gain stored in storage 114 and/or a user setting receivedvia user interface 116 (such as a volume adjustment). For a binauraldevice, each amplifier 108 may be calibrated with a predetermined gainfor the respective ear, such as based on a hearing (i.e., auditory)test. Thus, if the user has a hearing loss in one ear or differenthearing capabilities in each ear, the predetermined gain for eachamplifier 108 may be different. In some examples, amplifier(s) 108 mayalso apply different gains in different frequencies (of a frequencyband) to compensate for hearing loss in different frequencies. Thefrequency-dependent gains applied by amplifiers 108 (for a binauraldevice) may be the same or different, depending on the hearingcapability of each of the user's ears. In some examples, amplifier(s)108 (or acoustic detail manipulation unit 106) may include adigital-to-analog converter (DAC) for converting digital signal 134 toan analog signal 136.

Loudspeaker(s) 110 is capable of receiving modified signal 134 fromacoustic detail manipulation unit 106 and/or amplified signal 136 fromamplifier(s) 108. Loudspeaker(s) 110 may include any suitable transducercapable of converting the modified signal 134 (or amplified signal 136)into output acoustic signal 138, such that output acoustic signal 138 isprovided to the user's ear(s).

Controller 112 may be coupled to one or more of: microphone 102, audioinput interface 104, acoustic detail manipulation unit 106, amplifier(s)108, loudspeaker(s) 110, storage 114, user interface 116 and powersupply 118, to control the capture of auditory stream 132 (viamicrophone 102 or directly via audio input interface 104), controlauditory manipulation of auditory stream 132 and/or to control output ofthe modified auditory stream 134 (via amplifier(s) 108 andloudspeaker(s) 110). Controller 112 may include, for example, aconventional digital signal processor, a logic circuit or amicroprocessor. It is understood that one or more functions of acousticdetail manipulation unit 106 may be performed by controller 112. It willbe understood by one of skill in the art from the description hereinthat one or more of the functions of acoustic detail manipulation unit106, audio input interface 104, and amplifier(s) 108 may be implementedin software and may be performed by controller 112.

Storage 114 may be configured to store parameters for at least one ofaudio input interface 104, detail removal filter 120, noise generator122, speech recognizer 124 and speech synthesizer 126 (i.e.,predetermined acoustic detail characteristics of the user). Storage 114may also store parameters for at least one of audio input interface 104and amplifier(s) 108. Storage 114 may also store one or morepredetermined sound stimuli, for calibration of device 100 (includingthe user's predetermined acoustic detail characteristics). Storage 114may also store any user settings for device 100 (such as volumecontrol). Storage 114 may be a memory, a magnetic disk, a database oressentially any local or remote non-transitory, tangible device capableof storing data.

User interface 116 may include any suitable interface for capturingaudio stream 132, outputting modified audio signal 134, indicatingstorage, calibration of device 100 and/or display of quantities. Userinterface 116 may include any suitable user interface capable ofproviding parameters associated with one or more of audio inputinterface 104, acoustic detail manipulation unit 106 and amplifier(s)108. User interface 116 may further include an input device such as akeypad or touchpad for entering information. User interface 116 mayfurther include a display device for presenting information to the user.

Suitable embodiments of microphone 102, audio input interface 104,acoustic detail manipulation unit 106, amplifier(s) 108, loudspeaker(s)110, controller 112, storage 114, user interface 116 and power supply118 may be understood by the skilled person from the description herein.

Referring next to FIG. 2, a flowchart of an example method forcalibration of auditory manipulation device is shown. The example methodmay be used to calibrate parameters of device 200 associated with a userof device 100, based on the language processing capability of the user.At step 200, a user is connected to device 100 (FIG. 1), for calibrationof parameters of device 100. At step 202, the user is connected to abrainwave mapping acquisition device, such as an MEG acquisition device,in order to acquire brain responses (such as neuromagnetic data) duringthe calibration.

At step 204, one or more sound stimuli are selected, to determine theuser's language processing capability.

At step 206, the sound stimuli are presented to the user vialoudspeaker(s) 110 (FIG. 1) of device 100. At step 208, brain responsedata is acquired (via the brainwave mapping acquisition device)simultaneously with the presented sound stimuli (step 206). At step 210,the brain response data is analyzed to determine the language processingcapability of the user.

At step 212, acoustic detail characteristic(s) to be reduced aredetermined, and parameters for device 100 are selected (based on thedetermined characteristic(s), based on the processing capability of theuser (determined in step 210). The parameters are selected for detailremoval filter 120, noise generator 122, speech recognizer 124 and/orspeech synthesizer 126, to reduce a predetermined acoustic detailcharacteristic(s) of the sound stimuli. The selected parameters may beused for auditory manipulation of input audio; by filtering, theaddition of noise and/or use of a single speech production voice withpredetermined speech characteristics.

At step 214, the selected parameters (step 212) are applied to the soundstimuli (selected in step 204) to manipulate the sound stimuli, in orderenhance the user's language processing capability. At step 216, steps206-214 are repeated with the manipulated sound stimuli (step 214), andthe parameters of device 100 are adjusted until a predeterminedimprovement in language processing capability of the user is determined.For example, the uniqueness point and recognition point during a lexicalaccess. A uniqueness point and/or recognition point that occurs earlierin time (See FIG. 4B) may indicate that the manipulation improves thelanguage processing of the individual. At step 218, device 100 isprogrammed with the parameters determined in step 216.

At optional step 220, audiometric testing may be performed on the user.At optional step 222, device 100 may be further programmed based on theaudiometric testing results of step 220. For example, one or more gainparameters of amplifier(s) 108 may be selected based on the audiometrictesting results. In another example, calibration of device 100 may alsobe based on theoretical personalization.

FIGS. 3A and 3B illustrate example methods of auditory manipulation forenhancement of language processing, via acoustic detail manipulationunit 106 (FIG. 1). In particular, FIG. 3A is a flowchart diagram of anexample auditory manipulation of auditory stream 132 (FIG. 1) via detailreduction filtering and/or noise addition; and FIG. 3B is a flowchartdiagram of an example auditory manipulation of auditory stream 132 viaspeech recognition and speech synthesis.

Referring to FIG. 3A, at step 300, auditory stream 132 is received byacoustic detail manipulation unit 106. At step 302, an auditoryreduction method is selected, for example, via controller 112. Forexample, controller 112 may select detail removal filtering via filter120 and/or noise addition filtering via noise generator 122. At step304, parameters are selected for detail removal filter 120 and/or noisegenerator 122, for example, via controller 112 using filter/noiseparameters stored in storage 114.

At step 306, detail removal filtering and/or noise addition is appliedto auditory stream 132, via respective detail removal filter 120 andnoise generator 122. At step 308, modified signal 134 (after filteringand/or noise addition) is output from acoustic detail manipulation unit106. Although not shown in FIG. 3A, modified signal 134 may also beconverted to an analog signal and/or may be amplified by amplifier(s)108 before being transduced to output acoustic signal(s) 138 byloudspeaker(s) 110.

Referring to FIG. 3B, at step 310, auditory stream 132 is received byacoustic detail manipulation unit 106. At step 312, speech recognitionis applied to auditory stream 132, via speech recognizer 124. At step314, speech synthesis is applied to the recognition result (step 312),via speech synthesizer 126. Speech synthesizer 126 may select a speechproduction voice with predetermined speech characteristics. For example,the predetermined speech characteristics may be selected by controller112, using predetermined speech characteristics stored in storage 114.

At optional step 316, steps 302-306 may be applied, to apply detailremoval filtering and/or the addition of noise to the speech productionvoice (synthesized in step 314). At step 318, modified signal 134 (afterspeech synthesis in step 314 or after filtering/noise addition inoptional step 316) is output from acoustic detail manipulation unit 106.Similar to FIG. 3A, modified signal 134 may also be converted to ananalog signal and/or may be amplified by amplifier(s) 108 before beingtransduced to output acoustic signal(s) by loudspeaker(s) 110.

Auditory manipulation according to the present invention may be used toeliminate irrelevant acoustic detail, thus leading to stronger abstract(e.g., word, etc.) representations that may be accessed more rapidly inreal time. Thus, exemplary device 100 may reduce (or substantiallyeliminate) irrelevant detail from acoustically presented words. This mayaide the ASD brain to identify words with increased speed and accuracy,leading to more robust lexical representations for the ASD individual,with an overall positive effect on language and communication. Suchapproaches may be evaluated by similar electrophysiologic and behavioralassessments as described herein (e.g., MEG clustering, mismatch fieldand auditory processing latencies, as well as behavioral assays ofrepetition priming). Improvement in the categorization and abstractionby acoustic manipulation of stimuli may improve the user's lexicalsystem, by making it easier for the ASD brain to recognize words. Theseprocessing benefits may extend beyond lexical access into more abstractparts of language (such as decomposition of morphologically complexwords, syntactic processing and semantics).

Although the invention has been described in terms of devices andmethods for enhancing language processing in individuals with ASD, it iscontemplated that one or more steps and/or components may be implementedin software for use with microprocessors/general purpose computers (notshown). In this embodiment, one or more of the functions of the variouscomponents and/or steps described above may be implemented in softwarethat controls a computer. The software may be embodied in non-transitorytangible computer readable media (such as, by way of non-limitingexample, a magnetic disk, optical disk, flash memory, hard drive, etc.)for execution by the computer.

Although the invention is illustrated and described herein withreference to specific embodiments, the invention is not intended to belimited to the details shown. Rather, various modifications may be madein the details within the scope and range of equivalents of the claimsand without departing from the invention.

What is claimed:
 1. A method of auditory manipulation of an auditorystream of target speech for enhancement of language processing in anautism spectrum disorder (ASD) individual, the ASD individual havingbeen identified as having ASD based on a measured language processingcapability that is determined based on the ASD individual's brainwaveresponse to test speech, the measured language processing capabilitybeing insufficient to abstract perceptual objects from acousticallydistinct but conceptually identical words, the method comprising:receiving, by a processor, the auditory stream of the target speechperceptually representing an object; selecting an acoustic manipulationparameter for a predetermined acoustic detail characteristic associatedwith the ASD individual, the acoustic manipulation parameter computedbased on the measured language processing capability of the ASDindividual in response to the test speech; modifying, by the processor,the auditory stream of the target speech based on the selected acousticmanipulation parameter, to reduce the predetermined acoustic detailcharacteristic of the target speech while preserving a lexicality of thetarget speech, such that the reduced acoustic detail characteristicenhances perception of the object by the ASD individual when the targetspeech includes two or more acoustically distinct words eachperceptually representing the object; and outputting the modifiedauditory stream of the target speech to the ASD individual via at leastone loudspeaker.
 2. The method of claim 1, the method further including:capturing the auditory stream via a microphone proximate to the ASDindividual.
 3. The method of claim 1, the method further including:receiving the auditory stream from a remote device coupled to theprocessor, the remote device including at least one of a phone, acomputer, a television, and a playback device having an audiocapability.
 4. The method of claim 1, wherein the step of modifying theauditory stream includes applying a filter to the auditory stream toreduce the predetermined acoustic detail characteristic, the filterhaving a predetermined filter characteristic based on the selectedacoustic manipulation parameter.
 5. The method of claim 4, wherein thepredetermined filter characteristic includes at least one of a low-passfilter, a band-pass filter and a high-pass filter.
 6. The method ofclaim 1, wherein the step of modifying the auditory stream includessub-sampling the auditory stream at a predetermined sampling rate toreduce the predetermined acoustic detail characteristic, thepredetermined sampling rate based on the selected acoustic manipulationparameter.
 7. The method of claim 1, wherein the step of modifying theauditory stream includes adding noise to the auditory stream to reducethe predetermined acoustic detail characteristic, the noise having apredetermined noise characteristic based on the selected acousticmanipulation parameter.
 8. The method of claim 1, wherein the step ofmodifying the auditory stream includes: performing speech recognition onthe auditory stream to form a text representation of the audio stream;and converting the text representation of the auditory stream to aspeech production voice via speech synthesis processing, the speechproduction voice having a predetermined speech characteristic based onthe selected acoustic manipulation parameter.
 9. The method of claim 1,the method including, prior to receiving the auditory stream: presentingat least one predetermined stimulus to the ASD individual via the atleast one loudspeaker; acquiring one or more brain responses from theASD individual synchronous with the presented at least one predeterminedstimulus; determining the measured language processing capability of theASD individual based on the acquired one or more brain responses; anddetermining the predetermined acoustic detail characteristic to bereduced based on the measured language processing capability.
 10. Themethod of claim 9, wherein the one or more brain responses are acquiredfrom a magnetoencephalography (MEG) acquisition device.
 11. The methodof claim 1, wherein the predetermined acoustic detail characteristicincludes at least one of a pitch, a harmonic, an intonation, a transientsound, a sibilant sound or frequency dynamics of the target speech. 12.A device for auditory manipulation of an auditory stream of targetspeech for enhancement of language processing in an autism spectrumdisorder (ASD) individual, the ASD individual having been identified ashaving ASD based on a measured language processing capability that isdetermined based on the ASD individual's brainwave response to testspeech, the measured language processing capability being insufficientto abstract perceptual objects from acoustically distinct butconceptually identical words, the device comprising: an audio inputinterface configured to receive the auditory stream of the target speechperceptually representing an object; a non-transitory, tangible storagedevice configured to store acoustic manipulation parameters for apredetermined acoustic detail characteristic associated with the ASDindividual, the acoustic parameter computed based on the measuredlanguage processing capability of the ASD individual in response to thetest speech; an acoustic detail manipulation unit configured to: selectan acoustic manipulation parameter among the stored acousticmanipulation parameters for the predetermined acoustic detailcharacteristic, and modify the auditory stream of the target speechbased on the selected acoustic manipulation parameter, to reduce thepredetermined acoustic detail characteristic of the target speech whilepreserving a lexicality of the target speech, such that the reducedacoustic detail characteristic enhances perception of the object by theASD individual when the stimulus target speech includes two or moreacoustically distinct words each perceptually representing the object;and at least one loudspeaker configured to output the modified auditorystream of the target speech to the ASD individual.
 13. The device ofclaim 12, wherein the acoustic detail manipulation unit includes afilter configured to filter the auditory stream with a predeterminedfilter characteristic, the predetermined filter characteristic based onthe selected acoustic manipulation parameter.
 14. The device of claim12, wherein the acoustic detail manipulation unit includes a filterconfigured to sub-sample the auditory stream at a predetermined samplingrate, the predetermined sampling rate based on the selected acousticmanipulation parameter.
 15. The device of claim 12, wherein the acousticdetail manipulation unit includes a noise generator configured to addnoise having a predetermined noise characteristic to the auditorystream, the predetermined noise characteristic based on the selectedacoustic manipulation parameter.
 16. The device of claim 12, wherein theacoustic detail manipulation unit includes: a speech recognizerconfigured to convert the auditory stream to a text representation; anda speech synthesizer configured to convert the text representation to aspeech production voice, the speech production voice having apredetermined speech characteristic based on the selected acousticmanipulation parameter.
 17. The device of claim 12, wherein thepredetermined acoustic detail characteristic includes at least one of apitch, a harmonic, an intonation, a transient sound, a sibilant sound orfrequency dynamics of the stimulus.
 18. The device of claim 12, furthercomprising a microphone coupled to the audio input interface, themicrophone configured to capture the auditory stream.
 19. The device ofclaim 12, wherein the audio input interface is configured to receive theauditory stream from a remote device, the remote device including atleast one of a phone, a computer, a television, and a playback devicehaving an audio capability.
 20. The device of claim 12, wherein thedevice is configured to be calibrated based on at least onepredetermined stimulus presented to the ASD individual via the at leastone loudspeaker and one or more measured brain responses of the ASDindividual acquired synchronous with the presented at least onepredetermined stimulus.
 21. The device of claim 12, wherein the storagedevice is further configured to store one or more predetermined soundstimuli for calibration of the device.
 22. The device of claim 18,wherein the audio input interface further comprises an analog-to-digitalconverter for converting an audio signal from the microphone to adigital signal.